Editing digital images is a practice used by photography professionals and amateurs alike, given the wide availability of photo editing software. A type of editing common to all digital image processing is image sharpening. Image sharpening emphasizes the changes in an image; that is, it accentuates changes in a dark to light band as opposed to smoothing the change out across pixels.
There are limitations to image sharpening, however. Specifically, sharpening emphasizes “noise.” “Noise” in an image is the random fluctuation above and below actual image data and is a product of measuring imperfectly.
Unsharp masking is one kind of sharpening that is not too noise sensitive. Unsharp masking includes a smear phase and a differencing phase. Using the technique of unsharp masking produces a sharpened image based on a multiple of the difference between the pixel values in an original image and the corresponding pixel values of a blurred copy of the original image. Ordinarily, the blurred copy is computed by a convolution with a Gaussain, both of which are defined below.
Convolutions and Kernels
Convolution is a simple mathematical operation which is fundamental to many common image processing operators. Convolution provides a way of multiplying together two arrays of numbers, generally of different sizes, but of the same dimensionality, to produce a third array of numbers of the same dimensionality. The first array of numbers, in an image processing context, is the actual image, and the second array of numbers is a kernel. The kernel is applied against the image (added, multiplied, etc.) via convolution. Convolution is used in image processing to implement operators whose output pixel values are simple linear combinations of certain input pixel values.
By way of illustration, in an image processing context, one of the input arrays is simply a graylevel image. The second array is usually much smaller, and is also two-dimensional (although it may be just a single pixel thick), and is known as the kernel. The convolution is performed by sliding the kernel over the image, generally starting at the top left corner, so as to move the kernel through all the positions where the kernel fits entirely within the boundaries of the image. Each kernel position corresponds to a single output pixel, the value of which is calculated by multiplying together the kernel value and the underlying image pixel value for each of the cells in the kernel, and then adding all these numbers together.
Gaussain Blurs
The Gaussian smoothing operator is a two-dimensional convolution operator that is used to blur images and remove detail and noise. It uses a kernel that represents the shape of a Gaussian (“bell-shaped”) hump. The idea of Gaussian smoothing is to use this 2-D distribution as a point-spread function, and this is achieved by convolution. Since the image is stored as a collection of discrete pixels, a discrete approximation to the Gaussian function is produced before convolution is performed.
Gaussian blurs are an obvious choice for the smearing phase of unsharp masking, not because of its shape, but because it is, in theory, infinitely smooth. However, there are several problems with Gaussian blurs. First, true Gaussians have infinite extent, which is impossible in a kernel. Therefore, some approximation is required. The approximation must always include truncating the Gaussian kernel. A truncated Gaussian is no longer infinitely smooth, which is one of its critical characteristics. In addition to truncation, discretization (i.e. the process of converting from real values to discrete values) magnifies the non-smoothness of a truncated Gaussian kernel. When an image is convolved with a non-smooth kernel, any lack of smoothness of the blurring curve gets magnified in the differencing phase, which introduces noise into the image.
Second, Gaussian blurs take a relatively long time to compute, despite the availability of ever faster processors. Computation complexity is an issue because the size of digital images today generally begins around 5 mega pixels for amateur photo editors and can be as high as 15 mega pixels for professional photographers. Gaussian kernels are generally 5×5 matrices, which means that for each pixel in a graylevel image there might be 25 calculations, and for each pixel in a color image there might be 75 calculations (because the Gaussian blur is performed on each red, green, and blue pixel value in each pixel). Thus, for a 10 mega pixel color image, one Gaussian blur makes approximately 750,000,000 calculations. Such a strain on a processor is significant. As a result, many software applications, instead of performing all the required calculations, perform a shortcut by attempting to approximate Gaussian blurs, which results in even less of a smooth blurring and increases noise.
Lastly, Gaussian blurs do not work well at very small pixel radii, such as one pixel, which is most often what a user wants. There are several reasons for this. First, a Gaussian kernel must have the sum of its weight equal to 1. If it is very small-width kernel, the weights applied to individual pixels are large, which reduces smoothing. Second, the discretization error is proportionally much larger when a Gaussian kernel is approximated over only nine pixels, the minimum possible size (because the source kernel must be at the center and the kernel must be square, the kernel size must be the square of an odd number. A 1×1 kernel does nothing. Third, the undesirable effects of truncation are large because the individual pixel weights are large, so the dropoff to zero outside the kernel is sharp. Finally, the fundamental nature of a Gaussian blur is superposition, which roughly means blurring each pixel individually and summing the results. Thus, a Gaussian blur emphasizes precision over smoothness.
Based on the foregoing, there is room for improvement in image sharpening, particularly in the smearing phase of the process.
The techniques described in this section are techniques that could be pursued, but not necessarily techniques that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the techniques described in this section qualify as prior art merely by virtue of their inclusion in this section.